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In 2012 - 20 1 3, the Intelligent Robotics Group from NASA Ames 
Research Center is conducting 2 experiments with the International 
Space Station (ISS) 

Experiment I .‘Simulate an internal inspection of a module of the ISS using 
the free-fl/ing SPHERES robot with an Android Smartphone connected to 
it. 

Experiment 2: Simulate deployment of a telescope by having an astronaut 
on the ISS control the KIO Rover at NASA Ames. 

For both of these experiments, the 
astronauts will be using a custom 
“Workbench” RCP application. These 
are all based on Eclipse 3.7.2. 
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The ISS has a 450 page set of standards for software. This helps 
maintain consistency between various software control systems and 
helps astronauts with different native languages understand what 
various icons mean. 

We also have to deal with unique usability issues, such as the fact that 
it is very difficult to click and point with a mouse when you are in 
zero g. 

The only operating systems on the ISS computers with a GUI is 
currently Windows XP, so that is our target development platform. 
Internally we also use Linux and OSX, so we are doing cross-platform 
development. 

Since we are developing multiple RCP applications we put common 
code into shared plugins. (This is one of the reasons we are still on 
Eclipse 3.7.2) 
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Experiment I : SPHERES 



SPHERES are free-flying satellite robots 
typically used for orbital experiments. 
SPHERES have been on the ISS since 2006, 
and were developed at MIT (They do not 
include a Smartphone). 

They use a cold-gas C02 thruster system 
that is very similar to what is used on 
paintball guns.The entire system is powered 
using double-A batteries. A DSP 
microprocessor inside coordinates the 
mixing of the thrusters for the desired 
movement. The microprocessor also 
receives signals from five ultrasonic beacons, 
so the SPHERES can know where it is. 


The SPHERES microprocessor is already fully taxed with normal SPHERES operations; 
we needed to add more processing power and a camera. We determined the most 
efficient way to do this was to adapt a Smartphone to work with the SPHERES. 
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We had to remove the battery 
and power it with AA batteries, 
put teflon tape over the screen, 
and remove the GPS chip, as well 
as put it through rigorous 
testing. Naturally we use velcro 
along with a custom USB cable 
to connect it to the SPHERES. 
We upmassed it on the last 
shuttle launch. 

For the SPHERES experiments, 
we first controlled the SPHERES 
on the ISS from the SPHERES 
Smartphone Workbench RCP 
application running on Earth. 



In the next iteration of this experiment, an astronaut on the ISS will control the 
SPHERES using the SPHERES Smartphone Workbench. 
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When we are commanding and monitoring robots over this long of a distance, we 
have a time delay between when commands are sent and when they are received; we 
have to design our software to account for this. We also have to support loss of 
signal (LOS) times, when the ISS is unable to communicate with Earth. 

We use DDS, a data distribution system, to reliably send data. On computers, we use 
RTI’s implementation of DDS, with our own standards, called RAPID, running on top 
of that. On the Android we licensed CoreDX DDS libraries. 


The SPHERES Smartphone Workbenc (RCP application) talks to the Android 
Smartphone, which communicates via USB cable to send commands to the SPHERES, 
and report the state back to the SPHERES Smartphone Workbench. 
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Screenshot of the SPHERES Smartphone Workbench 



Neatx - tecohen@dale.ndc.nasa.gov:663 - dale.ndc.nasa.gov 


SPHERES Smartphone Workbench 


Ack 0 


Tip Verify connection by sending test echoes and reviewing im 


GPS 

SPHERES 


lljunl2 18:29:17 


Connected 


Verify Connection 


Network Status 


Preview Plans 


& Network Connected 

© Subscriber Match Yes 
& Local IP 


128.102.106.121 


Echoes 


Key 


18:03:00 1 test received 

18:03:00 0 test sent 

18:02:27 Subscribe to spheres_echo 


Run Plans 


Manual Control 


u Image Sensor 


MJI 


Image #1 


Stop SPHERES j 



-fr Setup 


Log 


?| Help 


Exit 


Completed estopdrift. 


rx 

VASA 


Eclipse RCP for ISS Experiments 











Screenshot of the SPHERES Smartphone Workbench: Previewing Plans 
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Screenshot of the SPHERES Smartphone Workbench: Running Plans 


SPHERES Smartphone Workbench 


File Help 


Run Plan Control 


1. Upload plan to SPHERES. 

Upload 


validl.splan 




2. Control plan on SPHERES. 


►► 


Plan Running validl 


Status Running 


Time 00:00:07 


Time 


Command 



Station 0 

© 00:00:03 

Segment 0-1 

© 

Station 1 

$ 00:00:03 

Orient 


Flashlight on 


Record start 


Segment 1-2 


Station 2 


Pause 


Segment 2-3 


u Image Sensor 


Mark Damage 


Image #1 


Completed orient. 


▲ 

▼ 

Ack 0 


GPS 

lljunl2 18:32:17 

Tip Connect to SPHERES and pause the plan to use manual coi 

SPHERES 

Connected 


Preview Plans 

Run Plans Manual Control 

Stop SPHERES j 

J£r Setup 


3D Live 



Top Side End Free Flip 



— POR 
FWD 


3D Live 



Top 

Side End Free Flip 



PORT 


L 


— Fwr 

OVHD 


C L °g 


?| Help 


^ Exit 


Af^SA 


Eclipse RCP for ISS Experiments 












Screenshot of the SPHERES Smartphone Workbench: Manual Control 
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How to change the layout of an Eclipse RCP application 

Here we are controlling the layout of the RCP application, and constructing the 
tabs (CTabFolder) which will control and respond to perspective switching. 

public class IssApplicationWorkbenchWindowAdvisor extends WorkbenchWindowAdvisor { 

^Override 

public void createWindowContents (Shell shell) { 

IWorkbenchWindowConf igurer configurer = getWindowConf igurer ( ) ; 

Menu menu = configurer . createMenuBar () ; 

shell . setMenuBar( menu) ; 

shell . setLayout ( new FormLayout ( ) ) ; 

m_topToolbar = configurer . createCoolBarControl ( shell ) ; 
m_perspectiveBar = createPerspectiveBarControl ( shell ) ; 
m_page = configurer . createPageComposite (m_cTabFolder ) ; 

m_perspectiveRegistry = configurer . getWindow( ) . getWorkbench ( ) . getPerspectiveRegistry ( ) ; 
createPerspectiveBarTabs ( ) ; 

m_rightToolbar = createRightToolbar ( shell ) ; 

mstatusline = new SimpleStatusLineManager () . createControl (shell) ; 

// The layout method does the work of connecting the controls together. 
layoutNormal ( ) ; 

} 
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Do the construction and customization of the CTabFolder 

protected Control createPerspectiveBarControl (Composite parent) { 
mcTabFolder = new CTabFolder (parent, SWT. TOP) { 
public int getBorderWidth ( ) { return 10; } 

}; 

setTabFolderFont (m_cTabFolder ) ; 
m_cTabFolder . setMinimumCharacter s (20); 
m_cTabFolder . setTabHeight (40); 
m_cTabFolder. setSimple( false) ; 
m_cTabFolder . setBorderVisible ( true) ; 

m_cTabFolder . setBackground ( ColorProvider . INSTANCE . WIDGET_BACKGROUND ) ; 

return m_cTabFolder ; 

> 


A method to create a tab 

protected CTabltem createTabItem( CTabFolder tabFolder, String title. 

Control control, final String id) { 

CTabltem tabltem = new CTabltem (tabFolder, SWT .NONE); 

tabltem. setText ( " " + title + " "); 

tabltem. setData( id) ; 
tabltem. setControl ( control ) ; 

return tabltem; 

} 

A method to select a perspective 

protected void selectPerspective (String perspectivelD, SelectionEvent e){ 

IWorkbenchPage page = m_workbenchWindow. getActivePage ( ) ; 
if (page != null) { 

IPerspectiveDescriptor descriptor = 

m_perspectiveRegistry . f indPerspectiveWithld ( perspectivelD ) ; 
page . setPerspective ( descriptor ) ; 
page . getActivePart ( ) . setFocus ( ) ; 

> 
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Set up the tabs based on defined perspectives 


protected void createPerspectiveBarTabs ( ) { 

for (String pelD : getPerspectiveExtensionlds ( ) ) { 

// automagically read the perspectives contributed by plugin. xml 

IConf igurationElement [ ] config = Platform. getExtensionRegis try ( ) . 

getConfigurationElementsForf "org. eclipse. ui" , "perspectives " , pelD ) / 
for ( IConf igurationElement e : config) { 

CTabltem item = createTabltem f m_cTabFolder , 

e . getAttr ibute ( " name " ) , m_page , e . getAttr ibute ( " id " ) _)j_ 


} 


} 


// have the tabs listen for selection and change perspective 

final CTabFolder tabFolder = m_cTabFolder ; 

m_cTabFolder . addSelectionListener (new SelectionListener ( ) { 
public void widgetSelected(SelectionEvent e) { 

CTabltem tabltem = tabFolder . getSelection( ) ; 

String perspectivelD = ( String) tabltem. getData( ) ; 
selectPerspective ( perspectivelD , e ) ; 
tabltem. getControl ( ) . setFocus ( ) ; 

} 

}); 

/ / have the tabs autochange if the perspective changes 
m_workbenchWindow. addPerspectiveListener (new PerspectiveAdapter ( ) { 
public void perspectiveActivated(IWorkbenchPage page, 

IPerspectiveDescriptor perspectiveDescriptor) { 

CTabltem foundTab = getTabForPerspective (perspectiveDescriptor. getld( )) ; 
if (foundTab != null) {m_cTabFolder. setSelection( foundTab) ; } 

} 

}); 

m_cTabFolder . setSelection ( 0 ) ; 

populateTopRightButtons (m_cTabFolder ) ; // this is how we contribute Stop SPHERES button 

m_cTabFolder ,pack( ) ; 

E. c np se rcp f or iss Experiments 
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Recording of the SPHERES experiment 
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Experiment 2: Surface Telerobotics 


Surface Telerobotics will examine how astronauts in the ISS can remotely 
operate a surface robot (KIO Rover) across short time delays. We will be 
simulating an astronaut teleoperating a rover on the lunar farside to deploy a 
low radio frequency telescope. 

The telescope is comprised of three arms made of Kapton polyimide film, 
which will rolled out behind the rover. 


Array 
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This will be the first time that an astronaut on the ISS will be 
controlling a sophisticated rover on Earth. 

The K 1 0 Rover has been used extensively for robotic and geologic 
field research. Our KIO rovers have been to numerous field sites on 
Earth including the Haughton Crater on Devon Island, Canada; Black 
Point Lava Flow, Arizona; and many sites in California. 

KIO has four-wheel drive, all wheel steering and a passive averaging 
suspension. The KIO rover’s navigational sensors include a GPS 
System, a digital compass, stereo hazard cameras, and an inertial 
measurement unit. KIO rovers run Rover Software, which supports 
autonomous navigation and obstacle avoidance. 



ISS 
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The KIO rover can be configured with different scientific instruments. For 
this experiment instruments include a custom panoramic camera (GigaPan), a 
rear-facing inspection camera to observe telescope deployment, a Velodyne 
to examine surface texture and to assess terrain hazards, and of course the 
film deployer. 


We control K 1 0 rover operations with “route plans” - a sequence of tasks 
that include stations, segments and tasks to do along the way. Rover 
software does its best to achieve the goals of the route plan, though if there 
is an obstacle along the way it may not succeed. 


We initially developed VERVE 
(discussed at EclipseCon 201 1) to 
allow rover engineers to visualize 
rover status in 3D within an Eclipse 
RCP application; the Surface 
Telerobotics Workbench includes some 
of the VERVE technology and plugins, 
and extends it to comply with the ISS 
standards. 
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Screenshot of the Surface Telerobotics Workbench: Running Plans 
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Screenshot of the Surface Telerobotics Workbench:Teleoperating the rover 
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How to fake buttons 

ISS standards require us to create unique rounded “command” buttons, so it is 
clear which buttons send important commands. These buttons draw images to 
a graphics context, and then render text over them. 

public class CommandButton extends Composite { 
public CommandButton (Composite parent , int style) { 
super (parent , SWT . NONE) ; 

GridLayout gl = new GridLayout ( 1 , false); 

gl .marginHeight = gl .marginWidth = gl . horizontalSpacing = gl . verticalspacing = 0; 
setLayout(gl) ; 

m gridData = new GridData(SWT. FILL, SWT, CENTER, true, false); 

m_gridData.widthHint = m_gridData .minimumWidth = m_width; 
m_gridData.heightHint = m_gridData .minimumHeight = m_height; 
setLayoutData(m_gridData) ; 
setSize (m_width, m_height); 

mbuttonLabel = new Canvas(this, SWT, NONE); 
m_buttonLabel . setSize (m_width, m_height) ; 
m_buttonLabel . setLayoutData (m_gridData ) ; 
m_buttonLabel . addPaintListener ( new PaintListener ( ) { 

public void paintControl (PaintEvent e) { draw(e.gc); } 

}); 


Enabled 



Pressed 
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Add listeners to the button and set its text 

m_buttonLabel . addListener ( SWT .MouseDown, new Listener ( ) { 


public void handleEvent (Event event) { 
if ( isEnabled ( ) ) { 

m_pressed = true; 

m_cur rent Image = m_pressedBgImage; 
m_buttonLabel . redraw ( ) ; 
m_buttonLabel . update ( ) ; 

} 

}}); 


m_buttonLabel . addListener ( SWT .MouseUp, new Listener () { 
public void handleEvent (Event event) { 
m_pressed = false; 
if ( isEnabled ( ) ) { 

m_cur rent Image = m_bglmage; 

for (SelectionListener listener : m_selectionListeners ) { 
listener .widgetSelected( new SelectionEvent (event) ) 

} 

} else { 

m_cur rent Image = m_disabledBgImage; 

} 

if (m_buttonLabel != null && !m_buttonLabel . isDisposed( ) ) { 

m_buttonLabel . redraw ( ) ; 
m_buttonLabel . update ( ) ; 

} 


} 


}); 


public void setText (String text){ 
m_textString = text; 
draw ( new GC (m_buttonLabel ) ) ; 
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Draw the button 


protected void draw(GC gc) { 

int imagey = Math . max ( 0 r (m buttonLabel . getSize ( ) .y - m_height) / 2) ; 

gc . drawlmage (m_currentlmage , 0, imagey); 

Color fg = isEnabled( ) PColorProvider . INSTANCE. black :ColorProvider. INSTANCE. darkGray; 
gc . setForeground ( f g ) ; 

Point size = gc.textExtent(m_textString) ; 

int x = Math. max (0, (m_width - size.x) / 2); 

int y = Math. max (0, (m_buttonLabel .getSize () .y - size.y) / 2) / 
if (m_pressed){ 

x +=3 ; 
y +=3; 

} 

gc . drawText (m_textString, x, y, true); 
gc. dispose ( ) ; 

} 


We could have gotten fancy with buttons made of multiple images which would 
stretch depending on the length of the text, but we didn’t. 
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Including log4j log messages in the Ul 



r 


15Febl3 13:30:30.710 Alert: Could not find terrain DEM and/or terrain i 

Ack 

1 


L - — ~ - 


The ISS standards require an error acknowledgement bar in a consistent place in 
the upper left. When important errors or alerts come in, there is an “Ack” button 
to the right that includes the number of unacknowledged messages. 

Users can then pop up the “log” view which shows the time, ack state and 
description of messages that came in. 


We use Apache’s log4j 
framework to log messages, 
and OSGi’s LogListener to 
read the messages in and 
display them in ourLogView. 
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How we get log4j messages into our Log View 

In our code, we just call logger.error(“message”) 


public class IssLogView extends ViewPart implements LogListener { 
public IssLogView() { 

setReader ( IssLogService. getlnstance ( ) . getLogReader ( ) ) ; 

mcomparator = new LogViewComparator ( ) ; // this lets us sort the way we want 

// add the logger appender to the Apache Log4j framework 

Logger . getRootLogger ( ) . addAppender ( new I ssLogger Appender ( ) ) ; 

Logger. getLogger ( IssLogger Appender . class . getName ( ) ) . setAdditivity (false ) / 

} 


protected class I ssLogger Appender extends AppenderSkeleton { 
protected void append (LoggingEvent event) { 

if ( event. getLevel( ) . isGreaterOrEqual (MIN_LEVEL) ) { 

String status = getEventLevelString ( event . getLevel () ) + 

event . getRenderedMessage ( ) ; 

String ds = LogViewUtils . con vertToCorrectDateFormat (event . getTimeStamp ( )) ; 

// convert this log message from the file to our IssLogEntry class , 

// and contribute it to the view to display in the table. 

processEntry (event. getLevel ( ) .toInt( ) , ds + " " + status); 
asyncRefresh(true) ; 

} 

} 

^ // simple class to hold log entries 

public class IssLogEntry { 
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protected String m_time; 
protected Level m_level; 
protected String m_description; 
protected boolean m_ack; 



Intelligent Robotics Group 
at NASA Ames Research Center 


• K 1 0 Rover among others 

• SPHERES 

• xGDS Ground Data Systems 

• VERVE 3D within Eclipse 

• Contributed the moon to Google Earth 

• Mars-o-vision (mars.planetary.org) 

• GigaPan robotic camera 

• GeoCam disaster response 

• Ames Stereo Pipeline 

• Vision Workbench 

• Tensegrity research 
... and more! 

http://irg.arc.nasa.gov 
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